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METHOD AND APPARATUS FOR CALIBRATING A TILED DISPLAY 
CROSS REFERENCE TO CO-PENDING APPLICATIONS 
BACKGROUND OF THE INVENTION 

5 

This invention relates to calibrating displays, and more particularly, to 
calibrating tiled projection displays that use multiple projectors to produce larger 
and/or higher resolution images. 

Multiple projector systems have been proposed and used for many years. In 

10 the 1950s, the "CINERAMA" system was developed for the film industry. The 
CINERAMA system projected three films using three separate projectors, which were 
combined to form a single panoramic image. Disneyland continues to use a similar 
multiple projector system. At Disneyland, a circle of projectors shines onto a screen 
that circles the wall of a round room. 

15 In the video field, multiple projector systems have been proposed and used for 

a number of specialty applications. For example, U.S. Patent No. 4,103,435 to 
Herndon and U.S. Patent No. 3,833,764 to Taylor suggest using multiple projector 
systems for flight simulators. In many of these systems, multiple video screens are 
placed next to each other to form a large image display for multiple projectors. A 

2 0 difficulty with many of the video based multiple projector display systems is making 
the multiple images appear as one single continuous image on the display screen. 

When two images are projected side-by-side on a single screen, there is 
normally a seam between the images. The final display image will either appear as 
two images placed side-by-side with a gap in between or, if the images are made to 

2 5 overlap on a single screen, there will be a bright line where the two images overlap. 

Because of the inconsistencies in conventional cameras, video processing, delivery 
channels, displays and, specifically, projectors, it is exceedingly difficult to perfectly 
match the resultant video images so that no tiling artifacts appear among the images. 
If the images are brought very close together on the same screen, there is typically 

3 0 both gaps and overlaps at each seam. 

The article entitled Design Considerations and Appl ications for Innovative 
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Display Options Using Projector Arrays , by Theo Mayer, SPIE Vol. 2650 (1996), pp. 
131-139, discloses projecting a number of discrete images in an overlapping relation 
and ramping the brightness of the discrete images in the overlapping regions of each 
image. Mayer discloses using a blending function to fade down each overlapping 
5 edge of the discrete images in such a way so as to compensate for the gamma (video 
signal reduction vs. light output curve) of a phosphor, light valve or LCD projector, 
with the goal of producing a uniform brightness across the display. 

U.S. Patent No. 5,136,390 to Inova et al. recognizes that the blending function 
typically cannot be a simple even ramping function. A typical video projector 

10 produces an image that becomes darker toward the edges of the image as a natural 
function of the lens system used, and has a number of bright and dark portions caused 
by normal irregularities in the signal, intermediate signal processor, projector, screen, 
etc. These inconsistencies typically vary from one video component to another, and 
even among different components with similar construction. Also, different types of 

15 projectors often respond differently to the same amount of brightness modification. 

Thus, a simple ramp of the brightness in the over-lapping regions can produced light 
and dark bands and/or spots in the resulting image. 

To overcome these limitations, Inova et al. suggest applying a simple even 
blending function to the overlapping regions of the image, as suggested by Mayer, but 

2 0 then manually tuning the simple even blending function at specific locations to 

remove the visible artifacts from the display. The location of each artifact is 
identified by manually moving a cursor over each location that is identified as having 
an artifact. Once the cursor is in place, the system tunes the corresponding location of 
the blending function so that the corresponding artifacts are removed. 
25 Since each artifact must be manually identified by a user, the process of 

calibrating an entire display can be time consuming and tedious. This is particularly 
true since many displays require periodic re-calibration because the performance of 
their projectors and/or other hardware elements tend to change over time. In view of 
the foregoing, it would be desirable to have a display that can be calibrated and re- 

3 0 calibrated with less manual intervention than is required by Inova et al. and others. 
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SUMMARY OF THE INVENTION 

The present invention overcomes many of the disadvantages of the prior art by 
providing a display that can be calibrated and re-calibrated with little or no manual 
intervention. To accomplish this, the present invention provides one or more cameras 
5 to capture an image on the display screen. The resulting captured image is processed 
to identify any non-desirable characteristics including visible artifacts such as seams, 
bands, rings, etc. Once the non-desirable characteristics are identified, an appropriate 
transformation function is determined. The transformation function is used to pre- 
warp the input video signal such that the non-desirable characteristics are reduced or 
10 eliminated from the display. The transformation function preferably compensates for 
spatial non-uniformity, color non-uniformity, luminance non-uniformity, and/or other 
visible artifacts. 

In one illustrative embodiment, a tiled projection display is provided that has 
two or more projectors arranged in an array configuration. The projectors may be 
15 direct write (e.g. CRT, LCD, DMD, CMOS-LCD) or any other type of projector. In a 
tiled type display, each of the projectors preferably projects a discrete image 
separately onto a screen, wherein the discrete images collectively form a composite 
image. The discrete images may or may not overlap one another. A camera is then 
directed at the screen to capture a capture image of at least a portion of the composite 

2 0 image. The capture image may encompass less than one tile, about one tile, the entire 

composite image, or any other portion of the composite image that is deemed 
desirable. 

A determining block then determines if the capture image has one or more 
non-desirable characteristics. The non-desirable characteristics may be determined by 
25 comparing the capture image, or a portion thereof, with a predetermined data set as 
more fully described below. The determining block is preferably provided in a 
processor or the like. In one illustrative embodiment, the processor resides in one 
location and services all projectors. In another illustrative embodiment, the processor 
function is physically distributed among the projectors. 

3 0 Once the non-desirable characteristics are determined, an identifying block 

identifies a transformation function that can be used to process the input video signal 
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and provide processed input video signals to selected projectors to reduce the non- 
desirable characteristics in the composite image. The non-desirable characteristics 
may include spatial non-uniformity, color non-uniformity, and/or luminance non- 
uniformity, but may also include other known image artifacts or irregularities. 
5 To determine the spatial distortion of the projection display, an input signal 

may be provided to selected projectors to project a number of discrete images, each 
exhibiting a predetermined pattern. The camera device can then be used to capture a 
capture image of at least a portion of the screen. Using the capture image, the 
distortion of the projection display can be determined by, for example, comparing the 

10 capture image with a predetermined expected image. Alternatively, or in addition to, 
the distortion can be determined by comparing the location of selected features of the 
predetermined pattern in adjacent discrete images, and more preferably, in selected 
overlapping regions between images. By using an affine, perspective, bilinear, 
polynomial, piecewise polynomial, global spline, or similar technique, a 

15 transformation function can be determined and applied to the input video signal to 
compensate for the spatial distortion of each projector. 

To determine the color and luminance distortion of the projection system, a 
number of input signals of varying intensity may be sequentially input to the 
projection display, wherein each input signal corresponds to a flat field image of a 

2 0 selected color. For example, a first input signal may correspond to a red flat field 
image having an LCD intensity of "255". The next input signal may also correspond 
to a red flat field image, but may have a LCD intensity of "220". Input signals having 
progressively lower intensity may be provided until the input signal has a LCD 
intensity of "0". This process may be repeated for both blue and green flat field 

25 images. The camera device preferably captures each of the flat field images, either as 
a single image if the field-of-view of the camera device corresponds to the entire 
display, or as multiple images if the camera device has a smaller field-of-view. The 
resulting images are preferably stored as an array of capture images. Once collected, 
the non-desirable characteristics of each capture image can be determined including 

30 the luminance domes for each projector. Thereafter, a transformation function may be 
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determined for reducing the luminance domes across selected tiles, and matching the 
brightness and color of each tile with adjacent tiles. 

It is contemplated that the camera device may be periodically activated to 
capture a new capture image. The determining block may then determine if the newly 
5 captured image has one or more non-desirable characteristics, as described above, and 
the identifying block may identify a new transformation function that can be used to 
process the input video signal and provide processed input video signals to selected 
projectors to reduce the identified non-desirable characteristics. Thus, it is 
contemplated that the present invention may also be used to periodically re-calibrate 
10 the display with little or no manual intervention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Other objects of the present invention and many of the attendant advantages of 
the present invention will be readily appreciated as the same becomes better 
15 understood by reference to the following detailed description when considered in 
connection with the accompanying drawings, in which like reference numerals 
designate like parts throughout the figures thereof and wherein: 

Figure 1 is a perspective view of a four-by-six array of projectors; 

Figure 2 is a perspective view of one illustrative projector of Figure 1; 

2 0 Figure 3 is a schematic diagram of an illustrative embodiment of the present 

invention with the field-of-view of the camera encompassing two or more tiles; 

Figure 4 is a block diagram showing an illustrative implementation for the 
processor block of Figure 3; 

Figure 5 is a schematic diagram of an embodiment similar to that shown in 
25 Figure 3, but with the field-of-view of the camera encompassing only about one tile; 

Figure 6 is a schematic diagram of an embodiment similar to that shown in 
Figure 3, but with the processing function of Figure 3 distributed among the 
projectors; 

Figure 7 is block diagram showing another embodiment of the present 

3 0 invention; 

Figure 8 is a flow diagram showing an illustrative method for calibrating a 
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display; 

Figure 9 is a flow diagram showing another illustrative method for calibrating 
a display, and in particular, a tiled display; 

Figure 10 is a flow diagram showing yet another illustrative method for 
5 calibrating a display, including distinguishing the distortion introduced by the camera 
from the distortion introduced by the rest of the display; 

Figure 11 is a diagram showing an illustrative pattern that is displayed and 
later captured for determining spatial distortions in the display; 

Figure 12 is a diagram showing the illustrative pattern of Figure 11 displayed 
10 on two adjacent and overlapping tiles, also for determining spatial distortions in the 
display; 

Figure 13 is a diagram showing the operation of an illustrative transformation 
function that can be used to reduce the spatial distortion in a display by moving 
selected features toward a corrective location; 
15 Figure 14 is a diagram showing the operation of an illustrative transformation 

function that may be used to reduce the spatial distortion in a display by moving 
selected features toward a corrective location by a distance that is related to a relative 
method, a weighted average for example, modified by composite image or global 
constraints; 

2 0 Figure 15 is a flow diagram showing an illustrative method for at least 

partially removing a spatial distortion from the display; 

Figure 16 is a flow diagram showing an illustrative method for identifying a 
transformation for a tiled display to at least partially removing a spatial distortion 
from the tiled display; 

2 5 Figure 17 is a graph showing the luminance domes for a LCD projector at 

various input intensities and showing how the dome shapes change depending on the 
input intensity level; 

Figure 18 is a schematic diagram showing the luminance domes for three tiled 
LCD projectors each at various input intensities; and 

3 0 Figure 19 is a flow diagram showing an illustrative method for at least 

partially removing a luminance distortion from the display. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention provides a tiled display that can be calibrated and re- 
calibrated with a minimal amount of manual intervention. To accomplish this, the 
5 present invention provides one or more cameras to capture an image of the display 
screen. The resulting captured image is processed to identify any non-desirable 
characteristics including visible artifacts such as seams, bands, rings, etc. Once the 
non-desirable characteristics are identified, an appropriate transformation function is 
determined. The transformation function is used to pre-warp the input video signal 

10 such that the non-desirable characteristics are reduced or eliminated from the display. 

The transformation function preferably compensates for spatial non-uniformity, color 
non-uniformity, luminance non-uniformity, and other visible artifacts. 

In one illustrative embodiment, a tiled display is provided that has two or more 
projectors arranged in an array configuration. The displays may be projection displays 

15 which use CRT, LCD, DMD, CMOS-LCD or any other type of imaging device, and 
may be front or rear projection types. In a tiled type display, each of the projectors 
preferably images or projects a discrete image separately onto a surface or screen, 
wherein the discrete images collectively form a composite image. The discrete 
images may or may not overlap one another. Such a configuration is shown in Figure 

20 1. 

An illustrative projector 8 is shown in Figure 2, and preferably uses one 
Digital Micromirror Device (DMD) 1 0. DMD devices typically include an array of 
electronically addressable, movable square mirrors that can be electro-statically 
deflected to reflect light. The use of a DMD device can provide a lightweight, 

2 5 reliable, digital display with a wide viewing angle and good picture clarity. Further, 

some DMD devices meet various MIL-STD-810 environmental and stress 
requirements, and can display color graphic, text and video data at various frame 
rates. 

The projector 8 also preferably includes various optical elements to properly 

3 0 prepare the incoming illuminations, illuminate the DMD 10, and project the outgoing 

image. As shown in Figure 2, the optical path may include two segments: the 
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illumination path 12 and the projection path 14. The optical path may start with a 
high-reliability, metal halide, short-arc lamp 16 that illuminates the DMD 10. The 
light from the arc lamp 16 passes through a rotating RGB color filter wheel 18. An 
illumination relay lens magnifies the beam to illuminate the DMD 10 and form a 
5 telecentric image at the DMD 10. A Total Internal Reflection (T1R) prism 20 enables 
the incoming light from the lamp to pass onto the DMD 10, and back into the 
projection optics. Depending on the rotational state (e.g. ±10 degrees for on/off) of 
each mirror on the DMD, the light from the DMD 10 is directed into the pupil of the 
projection lens (on) or away from the pupil of the projection lens (off). A multiple- 

10 element projection cell magnifies the image coming off the DMD 10, at the desired 
MTF, lateral color, and distortion. 

Each projector 8 may also include an electronics module (not explicitly 
shown). The electronics module may take the incoming data signals, convert the 
temporal signals into spatial representations on the DMD 10, and control the filter 18 

15 that provides the sequential color for the display. As described below, the electronics 
may be modular, allowing an arbitrary number of projectors to be tiled together. 
Further, tiling algorithms may be incorporated into the electronics, as appropriate, to 
enable "smart" projectors. This may allow the electronics of each projector to 
automatically or manually adapt to an arbitrary configuration of projectors, with little 

20 or no manual intervention by the user. 

Figure 3 is a schematic diagram of an illustrative embodiment of the present 
invention with the field-of-view of the camera encompassing two or more tiles. The 
system is generally shown at 50, and includes a processor 52, a first display which 
may be a projector 54, a second display which may be a projector 56, a viewing 

25 surface or a screen 58 and a camera 62. For the purpose of illustration the display will 
be discussed as a projector. The first and second projectors each project a discrete 
image onto the screen 58, as shown. The discrete images may be overlapping or non- 
overlapping, and may form a composite image on the screen 58. 

The processor 52 receives an input video stream 66. Because each of the 

3 0 projectors 54 and 56 project a portion of the desired composite image, the processor 
52 preferably segments the input video stream 66 into a first input video signal 72 and 
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a second input video signal 74. In the illustrative embodiment, the processor 52 
segments the input video stream so that any overlaps between adjacent discrete 
images, for example overlap 60, are taken into account as is known in the art. 

The input video stream 66 may be provided from any number of sources, and 
5 may be a NTSC, PAL, HDTV, workstation or PC video signal. These signal types are 
compatible with the RS-170 or RS-343 guidelines and specifications, for example, or 
more recently the VESA video signal standards and guidelines. The signals may 
include horizontal and vertical sync, and blanking information in addition to the active 
video signal used to build the output image. The sync signals may be used by the 

10 processor 52 to derive a system and/or video-sampling clock, especially in the' case of 
an analog input signal that needs to be digitized. 

Camera 62 is directed at the screen 58 as shown, and provides a camera output 
signal to the processor 52 via interface 64. The camera 62 may have a field-of-view 
that is sufficient to capture a capture image of at least a portion of the composite 

15 image. It is contemplated that the field-of-view may encompass only 1% of the 
composite image, 50% of the composite image, the entire composite image, or any 
other portion of the composite image that is deemed desirable. In a tiled display, this 
may correspond to only a portion of one tile, more than one tile, or all of the tiles. 
When the field-of-view of the camera does not encompass the entire display, it may 

2 0 be necessary to capture a capture image of each section of the display separately, and 

then assemble the results in a background or real-time mode to achieve a calibrated 
display over all the tiles. 

In the embodiment shown, the camera 62 has a field-of-view that is sufficient 
to encompass the discrete images provided by the first projector 54 and the second 
25 projector 56. The capture image is provided to the processor 52 as a feedback image 
via interface 64. A determining block, provided in processor 52, may determine if the 
capture image has one or more non-desirable characteristics. Preferably, the non- 
desirable characteristics are determined by comparing the capture image, or a portion 
thereof, with a predetermined data and information set as more fully described below. 

3 0 Once the non-desirable characteristics are determined, an identifying block, 

preferably within the processor 52, identifies a transformation function that can be 
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used to process the input video stream 66 and provide processed input video signals to 

projectors 54 and 56 which reduce the non-desirable characteristics in the composite 

image. The non-desirable characteristics may include spatial non-uniformity, color 

non-uniformity, and/or luminance non-uniformity, but may also include other known 

5 image artifacts or irregularities. 

It is contemplated that the projection display may be a front or rear projection 

display, and the camera 62 may be positioned in front of and/or behind the screen. In 

a second illustrative embodiment, the tiled projection display is a rear projection 

display having an array of tiled LCD type projectors, with each projector projecting a 

10 discrete image onto the back side of a transmissive screen 58. The transmissive 
screen 58 is viewed from the font side, and the camera 62 is used to capture a capture 
image of at least a portion of the screen 58 from the front side. 

It is contemplated that the camera 62 may be a still or video electronic camera, 
or have an equivalent combination of components that capture the scene in a multi- 

15 point manner and deliver an electronic representation of the image to the Processor 
52. In the preferred embodiment, the camera 62 is a CCD or CMOS camera, either 
color (e.g. multi-point colorimeter) or monochrome. The camera 62 preferably 
includes a photopic filter to enable the camera 62 to measure the output image in a 
manner that is consistent with the human visual system. Thus, noise and errors in 

20 luminance and chromaticity are measured in a way that is similar to how the eye 
detects such anomalies. The image may be a snapshot taken over a brief moment (e.g. 
less than 60 milliseconds), or over a longer exposure time (e.g. on the order of one 
second). 

In a preferred embodiment, the camera 62 may be a conventional camera 

2 5 device, such as a video miniature camera that produces an analog output. The analog 

output is digitized and captured by a frame grabber or the like located in the processor 
52. Once digitized the capture image can be stored and processed using digital 
processing techniques. To determine if the capture image has any non-desirable 
characteristics, the capture image may be compared to a predetermined data or 

3 0 information set. First, however, the distortion introduced by the camera 62 and 

associated processing hardware may be determined and removed. 
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To isolate the camera distortion, it is contemplated that a physical template 68 
may be provided in front of the screen 58, as shown. The physical template 68 
preferably includes a predetermined pattern thereon, such as an array of dots. With 
the physical template 68 in place, the camera 62 may capture a capture image of at 
5 least a portion of the physical template 68 including a portion of the predetermined 
pattern. By comparing the capture image with a predetermined expected image, and 
in particular, comparing the location of the dots of the predetermined pattern in the 
capture image to the expected locations of each of the dots, the distortion of the 
camera and associated hardware can be determined. Using the deviation from the 

10 expected locations, a transformation function can be determined and applied to the 
input video stream 66 to compensate for the camera distortion. 

After the camera distortion is determined, the physical template 68 may be 
removed, and the distortion of the display itself can be determined. The display may 
have a number of types of distortion including spatial distortion, color distortion, 

15 luminance distortion, etc. To determine the spatial distortion of the projection 
display, for example, an input signal may be provided to selected projectors 54 and 56 
to project a number of discrete images, each exhibiting a predetermined or known 
pattern. The camera 62 can then be used to capture a capture image of at least a 
portion of the screen 58. Using the capture image, the distortion of the projection 

2 0 display can be determined by, for example, comparing the capture image with a 
predetermined and/or expected image. Alternatively, or in addition to, the distortion 
can be determined by comparing the location of selected features of the predetermined 
pattern in adjacent discrete images, and more preferably, in selected overlapping 
regions 60 between images. By using an affine, perspective, bilinear, polynomial, 

2 5 piecewise polynomial, global spline or similar technique, a transformation function 

can be determined and applied to the input video stream 66 to compensate for the 
spatial distortion of the projectors 54 and 56. Preferably, the distortion introduced by 
the camera 62 is removed from the capture image, as described above, before the 
distortion of the projection system is determined. 

3 0 To determine the color and luminance distortion of the projection system, a 

number of input signals of varying intensity may be sequentially input to the 
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projection display, wherein each input signal corresponds to a flat field image of a 
selected color. For example, a first input signal may correspond to a red flat field 
image having an LCD input intensity of "255" or the brightest input value. The next 
input signal may also correspond to a red flat field image, but may have a dimmer 
5 LCD input intensity of "220". Input signals having progressively lower intensity may 
be provided until the input signal has a LCD input intensity of "0" or black, the 
dimmest input value. These inputs may be expressed as bright to dark equivalents 
especially if the input is an analog voltage instead of a digitally measured value. This 
process may be repeated for both blue and green flat field images. The camera 62 
10 preferably captures each of the flat field images, either as a single image snapshot 
taken periodically when the field-of-view of the camera 62 corresponds to the entire 
display, or as multiple images if the camera device has a smaller field-of-view. The 
resulting images are preferably stored as an array of capture images or compressed 
versions thereof in a memory within processor block 52. Once collected, the non- 
15 desirable characteristics of each capture image can be determined including the color 
corresponding and input intensity variant luminance domes of each projector 54 and 
56. 

Once the luminance domes are identified, a ceiling and floor may be 
determined for both color and intensity, across the entire display. For example, one 
2 0 projector may be brighter than another even though all are driven at a maximum 
intensity (e.g. LCD "255"), and the brightness provided by each projector may 
decrease near the edges of the image. Accordingly, a ceiling may be selected to match 
the dimmest super-positon result of all the tiles when all projectors are operated at 
maximum intensity. Likewise, a floor may be selected to match the brightest 

2 5 superposition result when all projectors are operated at minimum intensity (LCD "0"). 

Thereafter, a transformation function may be determined for reducing the 
luminance domes across selected tiles, and for matching the brightness and color of 
each tile with adjacent tiles. For example, the transformation function may be 
represented by a color look up table of captured or compressed color domes, a nearest 

3 0 neighbor detection and identification function and an interpolation function among the 
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nearest neighbors to determine the input level needed at the display to output the 
desired linear output level. 

In one embodiment, the transformation function makes the luminance 
variation across the entire display less than about two percent, which is less than one 
5 just-noticeable-difference (JND) according to Weber's Law. To help achieve this 
level of luminance uniformity, the transformation function is preferably a function of 
the X and Y location on the display, and for some image source technologies such as 
polysilicon LCDs, the LCD input intensity value. Preferably, the variations across the 
display are held to be less than one JND in accordance with the contrast modulation 

10 sensitivity curve of human vision. This curve allows more or less variation as a 
function of spatial frequency. 

When the display has overlapping tiles, it is contemplated that the distortion of 
the system may be directly determined from patterns projected on the display. For a 
tiled display having overlapping discrete images, a first feature may be identified in a 

15 selected overlapping region, wherein the first feature is projected by a first projector. 

Then, a second feature may be identified in the same selected overlapping region, 
wherein the second feature is projected by a second projector, and wherein the second 
feature corresponding to the first feature. The spatial relationship between the first 
and second features may then be determined, and a first transformation function for 

2 0 the first projector can be identified therefrom. Likewise, a second transformation 
function for the second projector can be identified. A further discussion of this can be 
found below with reference to Figures 12-14. 

Finally, it is contemplated that the camera 62 may be periodically activated to 
capture a new capture image. The determining block in processor 52 may determine 

2 5 if the newly captured image has one or more non-desirable characteristics, and the 

identifying block of the processor 52 may identify a new transformation function that 
can be used to process the input video stream 66 and provide processed input video 
signals to projectors 54 and 56 to reduce the identified non-desirable characteristics. 
Thus, it is contemplated that the present invention may be used to periodically re- 

3 0 calibrate the display with little or no manual intervention. The period of re-calibration 

may be increased or decreased as required by the operational environment. For 
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example, it may be done at a 60 Hz rate to negate effects in a high vibration 
environment. In a benign environment, such as may happen in a home, the period may 
be reduced to 0.001 Hz or less. 

It is also contemplated that processor 52 may include built-in-test logic. The 
5 built-in-self test logic may periodically detect if any portion of the display has failed, 
and if so, correcting for the failure by appropriately re-calibrating the display system. 
This is particularly useful when the discrete images overlap one another by about 50 
percent or more. The 50% value, as an example, demarcates a packing arrangement 
which is fully redundant, leading to significant fail-operational system attributes. Fail 

10 operational means that a component can fail but the system continues to be fully 
operational. With a 50% overlap, if one projector fails, at least one more is ready to 
fill in the void resulting in significant gains in system reliability. 

To save memory costs, the transformation functions, and the extracted 
features, information and data sets as described herein, are preferably represented and 

15 stored as a number of reduced information sets such as affine transformation or 
forward differencing coefficients or compression coefficients like those recommended 
in JPEG or MPEG specifications. Interpolation or the like can then be used to 
reconstruct the appropriate correction factors for any location among the selected 
points (see Figure 11 below). 

2 0 Figure 4 is a block diagram showing an illustrative implementation for the 

processor block 52 of Figure 3. In the illustrative embodiment, the processor block 
52 includes a first sub-processing block 80 for servicing the first projector 54, and a 
second sub-processing block 82 for servicing the second projector 56. An input video 
segmentoj block 84 segments the input video stream 66, and provides an appropriate 
25 segmented input video signal to the first sub-processing block 80 and the second sub- 
processing block 82. 

A Digitizer and Image Memory Block 86 receives the analog video signal 64 
from the camera 62 and converts it into digital form, typically an 8 bit value for red, 
an 8 bit value for green and another for blue. The output of the camera 62 can be a 

3 0 monochrome signal or color signal. If the output of the camera 62 is monochrome, 

the test images of separate red, green and blue values may be shown by the projector 
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from time to time and captured separately or in combination by the camera in 
combination with the Digitizer and Image Memory Block 86. The digitization 
function need not reside within the physical bounds of the processor. Rather, it may 
be a part of the camera itself The same is true of the Image Memory. Further, these 
5 88,64 and 86, preferably implemented in the apparatus of a CMOS camera, may be 
embedded in the hardware of the sub-processor block, 80. Further, these may all be 
embedded in a CMOS-LCD imaging device to achieve the highest level of 
integration. 

Because the camera 62 captures an image that corresponds to both projectors 
10 54 and 56 (see Figure 3), a camera segmentor block 88 may be provided to segment 
the capture image and provides the appropriate portions thereof to the first and second 
sub-processing blocks 80 and 82. 

The first sub-processing block 80 preferably has access to the captured and 
ideal images of the first projector 54 and the neighboring projectors, including the 
15 second projector 56. The capture image, or at least the appropriate portion thereof, is 
analyzed by the first sub-processing block 80. For spatial compensation, a number of 
features may be extracted from the image which may include seeking, detecting 
identifying, and extracting anchor points in the image. The anchor points may be, for 
example, features in a predetermined pattern (e.g. an array of dots) or may be deduced 
2 0 from the standard input video by deriving which features in the input image which are 
stochastically separable and uniquely identifiable. For color compensation, the 
projector(s) under test may project a series of images onto the screen ranging in 
intensity from LCD "0" to LCD "255", for red, green and blue separately. The camera 
62 may capture a color or monochrome image of each of the images on the screen. 

2 5 These capture images are preferably stored as an array in the Reference Images and 

Data block 90, which is implemented using storage media. Further, the red, green and 
blue compensation information may be obtained at initial setup and adjusted for 
example by scaling in real-time or periodically. This means the input video may be 
used to alter known detailed compensation data, thereby preferably circumventing the 

3 0 need to apply test images to learn what compensation needs to be applied over time 

and condition. 
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The vignetting aspects of the camera lens aperture and assembly may also be 
captured and included in the result. The vignette aspect of the camera can be 
measured ahead of time using a flat white field image provided by a uniformly 
illuminated white flat field and stored away as a priori information. This information 
5 may also be stored in compressed form in the Reference Images and Data block 90. 

The reference and measured imagery are compared in the Block Inverse 
Transform Calculator 100. The various test images, including spatial and color, are 
analyzed in this block. Salient and relevant features are extracted automatically 
preferably using variations of filter, threshold, linearity correction, and gamma 
10 correction methods. In order to obtain spatial compensation, the affine, perspective, 
bilinear, polynomial, piecewise polynomial, or global spline transformation, for 
examples, may be computed by comparing the measured spatial test pattern features 
with resident reference test image features. For color information, the gamma, gain 
and offsets of the camera, digitizer and projectors may be extracted. These and 
15 related features are categorized and solved to produce a set of spatial and color 
compensating transform coefficients. 

The transformation coefficients, calculated typically in a non-real-time mode, 
are loaded into the Real-time Warper and Color Blender block 102. This block 
converts the coefficients into high-speed real-time compensation signals which are 

2 0 provided to the first projector 52. The Transformed Video Signal 72 is preferably a 

pre-warped version of the Input Video Stream 66. The pre-warping can be local or 
global with respect to the tile and its neighbors. The pre-warping may be applied in 
color and space or other artifact dimension, time separation for example, in a manner 
that, when the Transformed Video Signal 72 is passed through the projector-screen 
25 system, the output image emerges in spatial -temporal and color alignment, with little 
or no visible artifacts. 

The Real-time Warper and Color Blender Block 102 can be implemented 
using a combination of standard processing components including high speed look-up 
tables, high speed digital signal processors, image memory, X, Y position counters, 

3 0 bilinear interpolation devices and/or forward differencing engines (made from 

coefficient registers, adders and latches, for example). 
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It is contemplated that that the alignment may be implemented in relative or 
absolute terms. For example, if alignment is done with respect to a physical template, 
this may be considered to be an absolute alignment. If, on the other hand, no physical 
template is used, and the behavior of the tiles is characterized relative to attributes of 
5 neighbor tiles, then this may be considered a relative alignment. 

The second sub-processor block 82 may be constructed in a similar manner. 
Because in some applications, the computation of a transformation function for one 
tile can depend on the information and transformation function of another tile, it is 
contemplated that an interface 106 may be provided between the first and second sub- 

10 processor blocks 80 and 82. This interface may allow the first sub-processing block 80 
to communicate with the second sub-processing block. While the processor block 52 
is shown having two separate sub-processing blocks 80 and 82, any number of other 
implementations are contemplated. For example, the processor block 52 may be 
implemented as an appropriately programmed general purpose microprocessor, an 

15 appropriately programmed digital signal processor, or any other implementation so 
long as the same or similar result is achieved. 

Figure 5 is a schematic diagram of an embodiment similar to that shown in 
Figure 3, but in this embodiment, the camera 120 has a field-of-view that 
encompasses only about one tile. In this configuration, the camera 120 may capture a 

20 series of images and its output used to determine a transformation function for the 
second projector 56, as described above. Subsequently, the camera 120 may be 
moved or its field of view moved using, for example, a deflecting mirror so that the 
field-of-view of the camera 120 encompasses the discrete image for the first projector 
54, as shown at 122. Then, the camera 120 may capture a series of images, as 

2 5 described above, and its output used to determine a transformation function for the 

first projector 54. This may be repeated until a transformation function can be 
determined for each projector in the display. 

Figure 6 is a schematic diagram of an embodiment similar to that shown in 
Figure 3, but with the processing function distributed among the projectors. As such, 

3 0 the first projector 54 and the second projector 56 each have a processor block 130 and 

132, respectively, associated therewith. 
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An Inter-Processor I/O is also included. These I/O channels may be 
implemented as video channels, parallel and/or serial data bus transmission lines, or 
any other type of communication link, including array position encoders or other array 
signature means. With the I/O function provided, the processors 130 and 132 form a 
5 distributed array of processors, potentially eliminating the need for a central executive 
processor. In one embodiment, processor 130 may assume the function of a global 
executive, processor 132 may assume the function of a color blend calculator, while 
another (not shown) may assume the function of a spatial warp calculator, and yet 
another may assume the function of a built in test monitor, etc. Preferably, each 

10 processor applies the same type of appropriate transformation to the corresponding 
portion of the input video stream to achieve a real-time transformation process. While 
an executive processor is not precluded, the Inter-Processor I/O 134 permits each tile 
cluster of resources to engage in dialog with its neighbors. This may be implemented 
as a local and global arrangement of information including image measurement and 

15 system compensation. The function of the processor array may identify the location of 
each tile, identify the neighboring tiles, and analyze the results including selected 
neighboring results. Accordingly, an arbitrary number and configuration of tiles may 
be provided enabling the tiling modules to be added or subtracted transparently by the 
user. 

20 Figure 7 is block diagram showing another embodiment of the present 

invention. An input image signal 140 is provided to a compensation device 142, 
where it is converted to or used as a reference signal image. Here, features may be 
extracted from the input image or, in the case of a calibration template, used as is. 
There the input signal is routed through to a video driver block 144. The signal is 

25 then provided to the Liquid Crystal Display (LCD) Driver 146. The LCD driver 
converts the input signal into special signals known in the art as required to drive the 
particular display device. The use of an LCD display is only illustrative. It is 
contemplated that the display device could be a DMD, ferroelectric, CRT or any type 
of electronic display. 

30 In the example shown, the projector 148 outputs an image signal as modulated 

light that provides a viewable image on the screen 150. There, the image is seen by 
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camera 152, which converts the image into an electronic signal. At each of these 
stages, distortion may be and typically is induced in the signal stream. The signal 
produced by the camera is then digitized by data translation digitizer block 154, 
converted into an image representation and compared to the reference signal image. 
5 The comparison is preferably done in terms of spatial and color image attributes. 

Thereafter, a transformation that corresponds to the distortion of the system is 
generated. To characterize the spatial distortion an 81 -point test pattern is provided 
(see, for example, Figures 11 and 12). The 81 -point test pattern is used to generate a 
set of global transformation coefficients or different sets of local transform 
10 coefficients. The models for the spatial distortion of one tile include, for example, the 
affine, perspective, bilinear, polynomial, piecewise polynomial, and global spline 
transformations. 

In accordance with the above, Figure 8 shows a flow diagram of an illustrative 
method for calibrating a display. The algorithm is entered at element 200, and control 
15 is passed to element 202. Element 202 captures a capture image of at least a portion 
of the composite image on a screen. Control is then passed to element 204. Element 
204 determines if the capture image has one or more non-desirable characteristics. 
Control is then passed to element 206. Element 206 identifies a transformation 
function that can be used to process an input video signal and provide a processed 

2 0 input video signal to selected projectors to reduce the non-desirable characteristics. 

Control is then passed to element 208, wherein the algorithm is exited. 

Figure 9 is a flow diagram showing another illustrative method for calibrating 
a display, and in particular, a tiled display. The algorithm is entered at element 220, 
wherein control is passed to element 222. Element 222 segments the input video 
25 * signal to identify a portion that corresponds to each tile of a tiled display. Control is 
then passed to element 224. Element 224 selects a first/next tile. Control is then 
passed to element 226. Element 226 applies a transformation to the portion of the 
input video signal that corresponds to the selected first/next tile by using the 
coefficients that correspond to the first/next tile and, through bi-linear interpolation or 

3 0 the like, producing a corresponding transformed input video signal. Control is then 

passed to element 228. Element 228 provides the transformed input video signal to 
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the projector(s) that correspond to the first/next tile. Control is then passed to element 
230. Element 230 determined if the selected first/next tile is the last tile in the 
display. If the selected first/next tile is not the last tile in the display, control is passed 
back to element 224. If, however, the selected first/next tile is the last tile in the 
5 display, control is passed to element 232, wherein the algorithm is existed. While the 
flow diagram shown in Figure 9 shows processing each of the tiles sequentially, it is 
contemplated that the tiles may be processed in parallel. 

Figure 10 is a flow diagram showing yet another illustrative method for 
calibrating a display, including distinguishing the distortion introduced by the camera 

10 from the distortion introduced by the rest of the display. The algorithm is entered at 
element 240, wherein control is passed to element 242. Element 242 provides a 
physical template adjacent to the screen. The physical template preferably includes a 
predetermined pattern. Control is then passed to element 244. Element 244 captures a 
capture image of at least a portion of the physical template using a camera device. 

15 Control is then passed to element 246. Element 246 determines a camera distortion 
that is introduced by the camera device by comparing the capture image with a 
predetermined expectation. Control is then passed to element 248. Element 248 
removes the physical template. Control is then passed to element 250. 

Element 250 provides an input signal to selected projectors to project a 

2 0 number of discrete images, each exhibiting a predetermined pattern. It is understood 

that only selected projectors may project a pattern, rather than all projectors. Control 
is then passed to element 252. Element 252 captures a capture image of at least a 
portion of the screen using the camera device. Control is then passed to element 254. 
Element 254 reduces or removes the distortion introduced by the camera from the 
25 capture image. Control is then passed to element 256. Element 256 determines a 
transformation function for reducing or removing the distortion introduced by the 
projection system by comparing the capture image with a predetermined expectation. 
Control is then passed to element 258, wherein the algorithm is exited. 

Figure 11 is a diagram showing an illustrative pattern with 9x9 dots that may 

3 0 be displayed and later captured for determining spatial distortions in a display. In the 

illustrative embodiment, each tile 268 is divided into eight segments across and eight 
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segments down resulting in 64 quadrilateral regions. The vertices of each region are 
the corresponding tie points. Accordingly, the tie points in the array of regions are 
used to determine the local distortion in region 270 and others across the tile 268. 
Different sets of local transformation coefficients correspond to different quadrilateral 
5 regions. The geometric distortion within each region is modeled by a transformation 
function governed by a pair of bilinear equation with eight degrees of freedom. The 
eight transformation coefficients are determined by comparing the known locations of 
the four tie points in the capture image to the corresponding expected locations as 
determined, for example, using a capture image of the template overlay. 

10 The appropriate correction factor for those locations that fall between the dots 

(for example, location 272) can be determined by using bilinear interpolation or the 
like. A further discussion of bilinear transformations can be found in Digital Image 
Warping , by George Wolberg, IEEE Computer Society Press Monograph, pages 50- 
51, which is incorporated herein by reference. A further discussion of spatial 

15 transforms can found in Digital Image Processing , 2nd edition, Refael C. Gonzalez 
and Paul Wintz, pages 246-251, which is also incorporated herein by reference. 

It is contemplated that the dot pattern need not be a regular lattice of points but 
may be derived by extracting stochastically reliable anchor points from snapshots of 
the incoming video stream captured in frame or image memory. These may be further 

2 0 correlated using auto and cross correlation algorithms, Bissels algorithm for example, 
which assimilates common points from a cloud of points viewed from different 
locations. 

Figure 12 is a diagram showing the illustrative pattern of Figure 11 displayed 
on two adjacent and overlapping tiles. A first tile is shown at 290 and a second tile is 

2 5 shown at 292. The first tile 290 and the second tile 292 are overlapping by a 

predetermined amount, as shown at 294. Each tile has a projector (not shown) for 
projecting a discrete image onto the corresponding tile. In the embodiment shown, 
each of the projectors is projecting a 9x9 array of dots. If the projectors were properly 
aligned, and there was no distortion in the system, each of the dots in the overlap 

3 0 region 294 would overlap one another. However, and as shown in Figure 12, if the 

projectors are not aligned the dots do not overlap one another. 
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To correct for this misalignment/distortion as detected by the camera and 
feedback system described herein, the present invention contemplates pre-warping the 
input video signal so that the corresponding dots properly align with one another. For 
example, the first projector, which corresponds to the first tile 290, projects dot 296 3 
5 and a second projector, which corresponds to the second tile 292, projects a 
corresponding dot 298. A first transformation function may be provided for 
effectively moving the location of the first dot 296 toward the second dot 298 when 
applied to the input signal of the first projector. Alternatively, or in addition to, a 
second transformation may be provided for effectively moving the location of the 

10 second dot 298 toward the first dot 296 when applied to the input signal of the second 
projector. If done properly, the first dot 296 and the second dot 298 overlap one 
another on the screen. Further, if done in accordance with the absolute or relative 
methods referred to above, then the compensated image is constrained globally and 
appropriately over the entire image. 

15 Using a relative compensation method, it is contemplated that the first 

transformation function may move the location of the first dot 296 toward the second 
dot 298 by an amount substantially equal to one-half the distance between the first 
and second dots. Likewise, the second transformation function may move the location 
of the second dot 298 toward the first dot 296 by an amount substantially equal to 

2 0 one-half the distance between the first and second dots. This is a straight averaging 

approach, devoid of global fit constraints like having to ensure that the implied grid 
lines are straight through the first derivative and equally spaced and is shown 
explicitly in Figure 13. 

Alternatively, or in addition to, the first transformation function may move the 
25 location of the first dot 296 toward the second dot 298 by an amount that is weighted 
by a predetermined blending function or some other factor at the first dot 296 relative 
to the second dot 298, and the second transformation function may move the location 
of the second dot 298 toward the first dot 296 by an amount that is weighted by a 
predetermined blending function or some other factor at the second dot relative to the 

3 0 first dot 296. This is a weighted average approach, and is shown explicitly shown in 

Figure 14. Preferably, the weighting function relates to the blending function used 
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for blending the color information of the tiles. This may be a ramp or spline or some 
other suitable function known in the art. 

When more than two corresponding dots must be considered, such as when 
three or more images overlap in a selected region, each of the corresponding dots may 
5 be moved toward a corrected location. This may be accomplished by using a similar 
averaging or weighted averaging approach, as discussed above. 

Other approaches are also contemplated. For example, it is contemplated that 
the transformation functions may maintain a predetermined relationship between 
selected dots. For example, dots 300, 302, 304 and 306 are from a common row of 

10 dots, and thus should fall along a common line 308. The transformation functions 
may maintain a linear relationship between these dots while still compensating for the 
distortion in the system. Likewise, dots 310, 312 and 314 are from a common column 
of dots, and thus should fall along a common line 316. The transformation functions 
may maintain a linear relationship between these dots while still compensating for the 

15 distortion in the system. Preferably, the linear relationship will provide continuity 
through the first derivative of the line functions and will preserve relatively uniform 
spacing among the implied connecting lines. 

In accordance with the above, Figure 15 is a flow diagram showing an 
illustrative method for at least partially removing a spatial distortion from the display. 

2 0 The algorithm is entered at element 330, wherein control is passed to element 332. 

Element 332 causes at least one of the projectors to project a discrete image that has a 
predetermined pattern with a number of features. Control is then passed to element 
334. Element 334 captures a capture image of a selected portion of the composite 
image. Control is then passed to element 336. Element 336 identifies a spatial 

2 5 distortion in the capture image by examining the relative location of selected features 

in the capture image. Control is then passed to element 338. Element 338 determines 
a transformation function that will at least partially remove the spatial distortion from 
the composite image. Control is then passed to element 340, wherein the algorithm is 
existed. Preferably, this method identifies the spatial distortion of the display by 

3 0 comparing projected image of a tile relative to the projected image of a neighbor tile, 

rather than or in addition to being relative to a physical template. 
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Figure 16 is a flow diagram showing a method for identifying a 
transformation for a tiled display to at least partially removing a spatial distortion 
from the tiled display. The algorithm is entered at element 350, wherein control is 
passed to element 352. Element 352 identifies a first feature in a selected overlapping 
5 region, wherein the first feature is projected by a first projector. This first feature may 
be extracted from a snapshot of the incoming video image. Control is then passed to 
element 354. Element 354 identifies a second feature in the selected overlapping 
region, wherein the second feature is projected by a second projector, and wherein the 
second feature corresponds to the first feature. Again, the second feature may be 

10 extracted from the incoming standard video input. Control is then passed to element 
356. Element 356 determines the spatial relationship between the first and second 
features, illustrative of establishing the relationship among an ensemble of features. 
Control is then passed to element 358. Element 358 identifies a first transformation 
function for the first projector. The first transformation function effectively moves 

15 the location of the first feature toward a corrective location when it is applied to the 
input signal of the first projector. Control is then passed to element 360. Element 360 
identifies a second transformation function for the second projector. The second 
transformation function, applied simultaneously with the first described, effectively 
moves the location of the second feature toward the corrective location when it is 

2 0 applied to the input signal of the second projector. Control is then passed to element 
362, wherein the algorithm is existed. The calculation of the correction function may 
be done periodically while the output is passed to the transformation function for real- 
time correction. 

In accordance with the present invention, the location of the dots may be 
2 5 determined by: subtracting a black capture image from the capture image that includes 
the dots; examining the resulting image content above a noise threshold using spatial 
filters which have an all-pass kernel; measuring the center of gravity of the dots to 
find the corresponding dot locations; eliminating dots whose energy threshold is 
below the threshold; sorting the dot locations for correlation with known or expected 
30 dot patterns and deriving corrective transformation functions therefrom. 

Figure 17 is a graph showing the luminance domes for one LCD projector 
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with various input intensities. As is evident, the magnitude of the luminance domes 
tends to increase as the input intensity increases. Also, the random brightness 
variation (e.g. noise) across the display tends to increases as the input intensity 
increases. This variation is attenuated and thus the signal to noise ratio augmented by 
filtering multiple time samples of dome capture images. This same general pattern is 
found for red, green and blue. In addition, each color typically has a different 
brightness value for the same input intensity. Further, in the case of polysilicon LCDs 
for example, the shape of the pattern changes as a function of the input intensity level, 
requiring the compensation function to attend to geometric and input intensity 
variables. 

To determine the color and luminance distortion of a projection system, and in 
particular, a tiled display system, direct view or projection, a number of input signals 
of varying input intensity may be sequentially input to the projection display. This 
may be done at initial calibration or periodically. Each input signal may correspond to 
a flat field image of a selected color. For example, a first input signal may correspond 
to a red flat field image having an LCD intensity of "255". The next input signal may 
also correspond to a red flat field image, but may have a LCD intensity of "220". 
Input signals having progressively lower intensity may be provided until the input 
signal has a LCD intensity of "0". This process may be repeated for both blue and 
green or other color flat field images. A camera device may capture each of the flat 
field images, either as a single image if the field-of-view of the camera device 
corresponds to the entire display, or as multiple images if the camera device has a 
smaller field-of-view. The resolution of the camera device may be chosen to be 
appropriate with the selected field-of-view. For example, when the field-of-view of 
the camera device is relatively wide, capturing an image of the entire display, a higher 
resolution camera device may be used. Likewise, when the field-of-view of the 
camera device is relatively narrow, capturing an image of only a small portion of the 
display, a lower resolution camera device may be used. In any case, the resulting 
images are preferably stored as an array of capture images or reduced resolution 
capture images or as compression coefficient capture images. Once collected, the 
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non-desirable characteristics of each capture image can be determined including the 
luminance or color domes for each projector. 

Once the luminance or color domes are identified, a ceiling and floor function 
which may be a linear or a spline or other suitable function are preferably determined 
5 for both color (including hue) and intensity, across the entire display. For example, 
one projector may be brighter than another at maximum intensity (e.g. LCD "255"), 
and the brightness provided by each projector may decrease near the edges of the 
image. Accordingly, a ceiling may be selected to match the dimmest superposition 
area of the tiles when all projectors are operated at maximum intensity. Likewise, a 

10 floor may be selected to match the brightest super-postion result of the tiles when all 
projectors are operated at minimum intensity (LCD "0"). 

Thereafter, a transformation function may be determined for compensating the 
luminance domes across selected tiles, and matching the brightness and color of each 
tile with adjacent tiles, thereby resulting in a linear display system. For example, the 

15 transformation function may be represented by a color look up table of captured or 
compressed color domes, a nearest neighbor detection and identification function and 
an interpolation function among the nearest neighbors to determine the input level 
needed at the display to output the desired linear output level. Preferably, the 
transformation function makes the luminance variation across the entire display less 

2 0 than about two percent for flat field test images, for example, which is less than one 

just-noticeable-difference (JND) according to Weber's Law. To help achieve this 
level of luminance uniformity, the transformation function is preferably a function of 
the X and Y location on the tile and of the input intensity level. 

Figure 18 is a schematic diagram showing the luminance domes for three-tiled 
25 LCD projectors each at various input intensities. A first tile 370, second tile 372 and 
third tile 374 each have different maximum brightness values for a common LCD 
input intensity, such as an input intensity of "255" as shown at 376, 378 and 380, 
respectively. The transformation function for each of the tiles preferably compensates 
the luminance domes across selected tiles using the reduced luminance dome 

3 0 information to linearize the dome (x, y and input intensity dependent) behavior of 

each tile in the system and using the blending function to allow for display tile 
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superposition. The transformation function also matches the hue of each tile with 
adjacent tiles, using the feedback image information in combination with known tri- 
stimulus color transformations or equivalent functions. For example, the 
transformation function for the first tile 370 may change the brightness in accordance 
5 with feedback behavior acquired at some time from the capture image of the first 
projector to lie along line 382 when the input intensity to the first projector has a 
value of "255" and when superposed over the output of tile 372. In the diagram 
shown, this may require that the center portion of the image be reduced more than the 
edge portions of the image because of the dome shape of the luminance profile. 

10 Likewise, the transformation function for the second tile 372 may change the 
brightness in accordance with feedback behavior acquired at some time from the 
capture image of the second projector to also lie along line 382 when the input 
intensity to the second projector has a value of "255" and when superposed with 
neighbor tile outputs. Finally, the transformation function for the third tile 374 may 

15 change the brightness of the third projector to lie along line 382 when the input 
intensity to the third projector has a value of "255" and in accordance with 
superposition with neighbor tile 372. 

As alluded to above, the transformation functions are also preferably 
dependent on the input intensity that is provided to the projectors. This is the result of 

2 0 the dependence of the capture image domes on the input intensity to image sources 

like the polysilicon LCD. 

For lower input intensities, the transformation functions may change the 
brightness of the first, second and third projectors to lie along, for example, lines 386 
or 388 as a function of the blending functions, the luminance dome dependence on the 
25 X, Y location on the screen and input intensity to the tiled display system. In this 
way, the transformation functions may make the luminance, hue, and saturation 
variation across the entire display relatively small regardless of the input intensity 
provided. That accomplished or enabled, the image content may be arbitrary making 
the tiled display with camera feedback suitable for displaying general imagery. 

3 0 In accordance with the above, Figure 19 is a flow diagram showing a method 

for at least partially removing a luminance distortion from the display. The algorithm 
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is entered at element 400, wherein control is passed to element 402. Element 402 
sequentially inputs one or more input signals that correspond to a flat field image of 
varying intensity to each projector. Control is then passed to element 404. Element 
404 captures a capture image of selected flat field images. Control is then passed to 
5 element 406. Element 406 identifies a luminance dome on one or more of the capture 
images. Control is then passed to element 408. Element 408 determines a feedback 
transformation function (dependent on X, Y and/or input intensity) that will at least 
partially remove the luminance domes from the composite image. Control is then 
passed to element 410, wherein the algorithm is existed. 

10 Finally, a method for determining the boundaries of each tile of a tiled display 

when using a camera that has a field-of-view that encompasses more than one tile is 
contemplated. This method includes displaying a white field image, for example, on 
all but a selected tile. The camera may then capture an image of display including the 
selected tile. Then it is a relatively simple matter to determine the boundaries of the 

15 selected tile by identifying the location where the white field image begins/stops. 

Another method is to display a checkerboard pattern wherein each of the tiles assumes 
one of two flat field images. In this embodiment, the boundaries for each tile may be 
determined by identifying the location where each of the flat field images 
begins/stops. Another method is to display a pattern of dots whose outer boundaries 

2 0 when detected by the camera in combination with a detection function define the 
boundaries of each tile as well. These may be used by the feedback processor-camera 
system to identify, among other things, the alignment of the projectors relative to one 
another. Further, and when the discrete images overlap one another, these methods 
further identify the extent of the overlap. 

2 5 Having thus described the preferred embodiments of the present invention, 

those of skill in the art will readily appreciate that the teachings found herein may be 
applied to yet other embodiments within the scope of the claims hereto attached. 
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CLAIMS 

1. In a system for controlling a tiled display system that receives an input 
video signal, the tiled display system having two or more displays, each manifesting 
one of a number of discrete images separately onto a viewing surface or screen to 

5 form a composite image, the improvement comprising: 

camera means for providing a capture image of at least a portion of the 
composite image; 

determining means for determining if the capture image has one or more non- 
desirable characteristics; and 
10 identifying means for identifying a transformation function that can be used to 

process the input video signal and provide processed input video signals to selected 
displays to reduce the non-desirable characteristics. 

2. A tiled display system according to claim 1, wherein the two or more 
15 displays are projection type displays. 

3. A tiled display system according to claim 1, wherein the two or more 
displays are front projection displays. 

2 0 4. A tiled display system according to claim 1, wherein the two or more 

displays are rear projection displays. 

5. A tiled display system according to claim 1, wherein the one or more 
non-desirable characteristics include spatial non-uniformity. 

25 

6. A tiled display system according to claim 1 , wherein the one or more 
non-desirable characteristics includes color non-uniformity. 



7. A tiled display system according to claim 1 , wherein the one or more 
3 0 non-desirable characteristics includes luminance non-uniformity. 
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8. A tiled display system according to claim 1, wherein the camera means 
is a colorimeter. 

9. A tiled display system according to claim 8, wherein the colorimeter 
5 comprises one or more video cameras. 

10. A tiled display system according to claim 1, wherein the tiled display 
system has a viewing side with said camera means located on the viewing side. 

10 11. A tiled display system according to claim 1 wherein the tiled display 

system has a viewing side with said camera means located on the opposite of the 
viewing side. 

12. A tiled display system according to claim 1, wherein said camera 
15 means periodically captures a new capture image during normal functional operation 

of the tiled display system. 

13. A system according to claim 12, wherein said determining means 
periodically determines if the new capture image has one or more non-desirable 

2 0 characteristics, and said identifying means periodically identifies a new 

transformation function that can be used to process the input video signal and provide 
processed input video signals to selected displays to reduce the non-desirable 
characteristics. 

25 14. A system according to claim 1, wherein said camera means captures 

the capture image during a calibration period. 

15. A tiled display system according to claim 1, wherein the camera means 
captures a portion of the composite image that substantially corresponds to one of the 

3 0 number of discrete images. 
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16. A system according to claim 1 , wherein the camera means sequentially 
captures a portion of the composite image, wherein each sequentially captured portion 
substantially corresponds to one of the number of discrete images. 

5 17. A method for controlling a display system that has two or more 

displays, each display manifesting one of a number of discrete images separately onto 
a viewing surface or screen to form a composite image, the method comprising the 
steps of: 

capturing a capture image of at least a portion of the composite image; 

1 o determining if the capture image has one or more non-desirable characteristics; 

and 

identifying a transformation function that can be used to process an input 
video signal and provide a processed input video signal to selected displays to reduce 
the non-desirable characteristics. 

15 

18. A method according to claim 17, wherein said capturing, determining 
and identifying steps are repeated periodically during normal functional operation of 
the projection display. 

2 0 19. A method according to claim 17, wherein said determining step 

compares the capture image with a predetermined data set to determine if the capture 
image has one or more non-desirable characteristics. 

20. A method according to claim 17, wherein said transformation function 
25 is determined during a calibration procedure. 

21. A method according to claim 20, wherein said calibration procedure 
includes the steps of: 

sequentially inputting one or more input signals that correspond to a flat field 

3 0 image of varying intensities; 

capturing a capture image of selected flat images; 
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identifying a luminance dome in the capture images; and 

determining a transformation function that will at least partially remove the 
luminance domes from the composite image. 

22. A method according to claim 21 , wherein the transformation function 
is represented by a color look up table of captured color domes, a nearest neighbor 
detection and identification block and an interpolation block among the nearest 
neighbors to determine the input level needed at the display to output the desired 
linear output level. 

23. A method according to claim 22, wherein the transformation function 
is represented, at least in part, by a number of coefficients. 

24. A method according to claim 22, wherein the transformation function 
is a bilinear transformation function. 

25. A method according to claim 20, wherein said calibration procedure 
includes the steps of: 

causing the at least one of the two or more displays to manifest a discrete 
image that has a predetermined pattern with a number of features; 

capturing a capture image of a selected portion of the composite image; 

identifying a spatial distortion in the capture image by examining the relative 
location of selected features in the capture image; 

determining a transformation function that will at least partially remove the 
spatial distortion from the composite image. 

26. A method according to claim 25, wherein said predetermined pattern 
includes a dot pattern. 

27. A method according to claim 26, further comprising the steps of: 
locating the dots in the capture image; and 
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identifying a transformation function that will at least partially remove the 
spatial distortion from the composite image by comparing the location of the dots in 
the capture image relative to an expected location.. 

5 28. A method according to claim 27, wherein said locating step includes 

the steps of: 

calculating the dot locations by subtracting a black capture image from the 
capture image that includes the dots; 

examining the resulting image content above a noise threshold using spatial 
10 filters which have an all-pass kernel; 

measuring the center of gravity of the dots to find the corresponding dot 
locations; 

eliminating dots whose energy threshold is below the threshold; 
sorting the dot locations for correlation with known or expected dot patterns 
15 and deriving corrective transformation functions therefrom. 

29. A method for calibrating a tiled display system, the tiled display 
system having two or more displays, each receiving an input signal and each 
manifesting one of a number of discrete images separately onto a viewing surface or 
2 0 screen to form a composite image, the method comprising the steps of: 

providing an input signal to selected projectors to project a number of discrete 
images, each discrete image having a predetermined pattern with a number of 
corresponding features; 

capturing a capture image of at least a portion of the screen using a camera 

25 device; 

determining a distortion in the composite image by isolating the location of 
selected features in the capture image; 

identifying a transformation function that can be applied to the input signal of 
selected displays to at least partially remove the distortion from the composite image. 

30 
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30. A method according to claim 29, wherein each display projects one of 
the number of discrete images separately onto a screen in an overlapping manner to 
form a composite image having one or more overlapping regions. 

31. A method according to claim 30, wherein at least one of the 
overlapping regions corresponds to more than 25 % of the corresponding discrete 
images. 

32. A method according to claim 30, wherein at least one of the 
overlapping regions corresponds to more than 50 % of the corresponding discrete 
images. 

33. A method according to claim 30, wherein said determining step 
includes the steps of: 

identifying a selected feature in a selected overlapping region, wherein the 
selected feature is manifested by a selected display; 

identifying a selected feature in the selected overlapping region, wherein the 
selected feature is correlated to a corresponding feature; 

determining the spatial relationship among the selected features; and 

identifying one or more transformation functions for the display, said 
transformation functions effectively moving the location of the selected features on 
the viewing surface or screen toward a corrective location when applied to the input 
signal of the selected display. 

34. A method according to claim 33, wherein said first transformation 
function effectively moves the location of the selected features toward the corrective 
location by an amount that is weighted by a blending function of the tile exhibiting 
the selected feature. 

35. A method according to claim 33, wherein the predetermined pattern 
has a predetermined linear relationship among selected features. 
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36. A method according to claim 35, wherein said first transformation 
function effectively moves the location of the selected feature toward the 
corresponding feature, and said second transformation function effectively moves the 

5 location of the corresponding feature toward the selected feature, while maintaining 
the predetermined linear relationship between the features. 

37. A method for calibrating a tiled projection display, the tiled projection 
display having two or more projectors, each receiving an input signal and each 

10 projecting one of a number of discrete images separately onto a screen to form a 
composite image, the method comprising the steps of: 

sequentially inputting to selected projectors an input signal that correspond to 
a flat field image of a first color for each of a number of luminance intensities; 
capturing a capture image of selected flat field images; 
15 determining a distortion in the projection display by examining the captured 

flat field images; and 

identifying a transformation function that can be applied to the input signal 
that will at least partially remove the distortion from the composite image. 

2 0 38. A method according to claim 37, wherein the inputting, capturing, 

determining and identifying steps are repeated for each of two or more colors. 

39. A method according to claim 37, wherein said projection display 
includes a number of pixels, and said determining step determines a distortion using 

2 5 more than one of the pixels. 

40. A method according to claim 39, wherein said determining step 
determines an X-distortion by examining more than one pixel in an X-direction, and a 
Y-distortion by examining more than one pixel in a Y-direction, so that an X-Y 

3 0 dependence of the distortion can be determined. 
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41. A method according to claim 40, wherein said identifying step 
identifies the transformation function by examining the X-Y dependence of the 
distortion determined by the determining step. 

5 42. A method for calibrating a tiled projection display, the tiled projection 

display having two or more projectors, each receiving an input signal and each 
projecting one of a number of discrete images separately onto a screen to form a 
composite image, the method comprising the steps of: 

providing a physical template that includes a predetermined pattern adjacent 
1 0 the screen; 

capturing a capture image of at least a portion of the physical template using a 
camera device; and 

determining a camera distortion that is introduced by the camera device by 
comparing the capture image with a predetermined expectation. 
15 43. A method according to claim 42, further comprising the steps of: 

removing the physical template; 

providing an input signal to selected projectors to project a number of discrete 
images, each exhibiting a predetermined pattern; 

capturing a capture image of at least a portion of the screen using a camera 
2 0 device; and 

determining a projection system distortion that is introduced by the projection 
system by comparing the capture image with a predetermined expectation. 

44. A method according to claim 43, further comprising the step of 
2 5 removing the camera distortion before determining the projection system distortion. 
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